Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures

Reinforcement learning (RL) constitutes a promising solution for alleviating the problem of traffic congestion. In particular, deep RL algorithms have been shown to produce adaptive traffic signal controllers that outperform conventional systems. However, in order to be reliable in highly dynamic urban areas, such controllers need to be robust with the respect to a series of exogenous sources of uncertainty. In this paper, we develop an open-source callback-based framework for promoting the flexible evaluation of different deep RL configurations under a traffic simulation environment. With this framework, we investigate how deep RL-based adaptive traffic controllers perform under different scenarios, namely under demand surges caused by special events, capacity reductions from incidents and sensor failures. We extract several key insights for the development of robust deep RL algorithms for traffic control and propose concrete designs to mitigate the impact of the considered exogenous uncertainties.Comment: 8 page

Deep-seeded Clustering for Unsupervised Valence-Arousal Emotion Recognition from Physiological Signals

Emotions play a significant role in the cognitive processes of the human brain, such as decision making, learning and perception. The use of physiological signals has shown to lead to more objective, reliable and accurate emotion recognition combined with raising machine learning methods. Supervised learning methods have dominated the attention of the research community, but the challenge in collecting needed labels makes emotion recognition difficult in large-scale semi- or uncontrolled experiments. Unsupervised methods are increasingly being explored, however sub-optimal signal feature selection and label identification challenges unsupervised methods' accuracy and applicability. This article proposes an unsupervised deep cluster framework for emotion recognition from physiological and psychological data. Tests on the open benchmark data set WESAD show that deep k-means and deep c-means distinguish the four quadrants of Russell's circumplex model of affect with an overall accuracy of 87%. Seeding the clusters with the subject's subjective assessments helps to circumvent the need for labels.Comment: 7 pages, 1 figure, 2 table

Vehicle Tracking Using the k-shortest Paths Algorithm and Dual Graphs

Vehicle trajectory descriptions are required for the development of driving behavior models and in the calibration of several traffic simulation applications. In recent years, the progress in aerial sensing technologies and image processing algorithms allowed for easier collection of such detailed traffic datasets and multiple-object tracking based on constrained flow optimization has been shown to produce very satisfactory results, even in high density traffic situations. This method uses individual image features collected for each candidate vehicle as criteria in the optimization process. When dealing with poor image quality or low ground sampling distances, feature-based optimization may produce unreal trajectories. In this paper we extend the application of the k-shortest paths algorithm for multiple-object tracking to the motion-based optimization. A graph of possible connections between successive candidate positions was built using a first level criteria based on speeds. Dual graphs were built to account for acceleration-based and acceleration variation-based criteria. With this framework both longitudinal and lateral motion-based criteria are contemplated in the optimization process. The k-shortest disjoints paths algorithm was then used to determine the optimal set of trajectories (paths) on the constructed graph. The proposed algorithm was successfully applied to a vehicle positions dataset, collected through aerial remote sensing on a Portuguese suburban motorway. Besides the importance of a new trajectory dataset that will allow for the estimation of new behavioral models and the validation of existing ones, the motion-based multiple-vehicle tracking algorithm allowed for a fast and effective processing using a simple optimization formulation. Keywords: vehicle trajectories; image processing; driver behaviour; remote sensing

Cytotoxic effects of cyanoacrylates used as retrograde filling materials: an in vitro analysis

Os cianoacrilatos tem encontrado aplicabilidade tanto na Medicina como na Odontologia há muitos anos. Têm sido usados como curativo após exodontias, bem como para obturação retrógrada em cirurgia parendodôntica. O objetivo deste estudo foi o de avaliar o efeito citotóxico do Histoacryl e outros dois homólogos etil cianoacrilatos: SuperBonder e Ultrabond, em cultura de fibroblastos, empregando ensaios de viabilidade pela exclusão de células coradas pelo azul de Trypan. Os cianoacrilatos foram aplicados em lamínulas de vidro circulares, que foram colocadas sobre cultura de fibroblastos NIH - 3T3. Após 0, 6, 12 e 24 horas (resposta celular imediata - curto prazo) e 1, 3, 5 e 7 dias (sobrevivência celular - longo prazo), foram efetuadas contagens celulares em microscópio de fase. Os dados obtidos foram analisados valendo-se do teste estatístico de Kruskal-Wallis. No experimento de curto prazo, somente as culturas do grupo Ultrabond (GIV) apresentaram porcentagens de viabilidade celular significantemente menores que a dos outros grupos (GI: controle; GII: Super Bonder; GIII: Histoacryl). Embora as culturas do grupo Super Bonder (GII) apresentassem porcentagens de viabilidade celular menores que as dos outros grupos (GI, GIII, GIV) no experimento de longo prazo, esse grupo foi o único que mostrou crescimento celular progressivo e contínuo. Nossos resultados mostraram biocompatibilidade in vitro tanto do Histoacryl como dos outros dois homólogos etil cianoacrilatos. Esses cianoacrilatos podem ser importantes para finalidades biológicas.Cyanoacrylate has been used in medicine and dentistry for many years. It has been used as a postextraction dressing and retrograde filling material in endodontic surgery. The aim of this study was to evaluate the cytotoxic effects of Histoacryl and other two homologue ethyl cyanoacrylates, Super Bonder and Ultrabond, on cultured fibroblasts, using the Trypan blue dye exclusion assay. The cyanoacrylates were applied to round glass coverslips, which were placed in contact with NIH 3T3 cells. After 0, 6, 12 and 24 h (short-term assay; viability) and 1, 3, 5 and 7 days (long-term assay; survival), the cells were examined under phase light microscopy and counted. The data were compared by the Kruskal-Wallis test. In the short-term experiments, only the cultures of the Ultrabond group (GIV) presented significant smaller percentages of cell viability than the cultures of the other groups (GI: control; GII: Super Bonder; GIII: Histoacryl). Although the cultures of the Super Bonder group (GII) presented smaller percentages of cell viability than cultures of the other groups (GI, GIII, GIV) at the long-term assay, this group was the only experimental group presenting a continuous and progressive cell growth. Our results have shown an in vitro biocompatibility of Histoacryl and ethyl cyanoacrylate homologues. These cyanoacrylates could therefore be of importance for endodontic purposes

Machine learning til realtidsforudsigelser af oprindelse-til-destination efterspørgsel for jernbaner med smart card og udbudsdata : Udvidet resumé

Realtidsforudsigelser af passagerefterspørgsel på jernbanen kan bidrage til smartere trafikstyring og på sigt til at udvikle et offentligt transportsystem som på forskellig vis imødekommer ekstraordinær efterspørgsel. Dette kræver adgang til detaljeret information om efterspørgselsmønstre i form af løbende indsamling af passagertal for hvert par af oprindelses- og destinationsstationer i korte tidsintervaller. I dette studie udvikles en machine learning model til forudsigelser af afvigelser fra det periodiske efterspørgselsmønster på Københavns S-bane i 15 minutters intervaller ved hjælp af realtidsdata fra Rejsekortet på efterspørgselssiden og Banedanmarks driftsstatistikker på udbudssiden. Studiet belyser dels betydningen af udbud for forudsigelse af efterspørgsel og dels udforskes måden hvorpå spatiotemporal data indlejres i modeller fra dyb læring for at opnå nøjagtige forudsigelser for mange-dimensionale og sparsomme data som disse

W–SPSA in Practice: Approximation of Weight Matrices and Calibration of Traffic Simulation Models

AbstractThe development and calibration of complex traffic models demands parsimonious techniques, because such models often involve hundreds of thousands of unknown parameters. The Weighted Simultaneous Perturbation Stochastic Approximation (W–SPSA) algorithm has been proven more efficient than its predecessor SPSA (Spall, 1998), particularly in situations where the correlation structure of the variables is not homogeneous. This is crucial in traffic simulation models where effectively some variables (e.g. readings from certain sensors) are strongly correlated, both in time and space, with some other variables (e.g. certain OD flows). In situations with reasonably sized traffic networks, the difference is relevant considering computational constraints. However, W–SPSA relies on determining a proper weight matrix (W) that represents those correlations, and such a process has been so far an open problem, and only heuristic approaches to obtain it have been considered.This paper presents W–SPSA in a formally comprehensive way, where effectively SPSA becomes an instance of W–SPSA, and explores alternative approaches for determining the matrix W. We demonstrate that, relying on a few simplifications that marginally affect the final solution, we can obtain W matrices that considerably outperform SPSA. We analyse the performance of our proposed algorithm in two applications in motorway networks in Singapore and Portugal, using a dynamic traffic assignment model and a microscopic traffic simulator, respectively

W–SPSA in Practice: Approximation of Weight Matrices and Calibration of Traffic Simulation Models

The development and calibration of complex traffic models demands parsimonious techniques, because such models often involve hundreds of thousands of unknown parameters. The Weighted Simultaneous Perturbation Stochastic Approximation (W-SPSA) algorithm has been proven more efficient than its predecessor SPSA (Spall, 1998), particularly in situations where the correlation structure of the variables is not homogeneous. This is crucial in traffic simulation models where effectively some variables (e.g. readings from certain sensors) are strongly correlated, both in time and space, with some other variables (e.g. certain OD flows). In situations with reasonably sized traffic networks, the difference is relevant considering computational constraints. However, W-SPSA relies on determining a proper weight matrix (W) that represents those correlations, and such a process has been so far an open problem, and only heuristic approaches to obtain it have been considered. This paper presents W-SPSA in a formally comprehensive way, where effectively SPSA becomes an instance of W-SPSA, and explores alternative approaches for determining the matrix W. We demonstrate that, relying on a few simplifications that marginally affect the final solution, we can obtain W matrices that considerably outperform SPSA. We analyse the performance of our proposed algorithm in two applications in motorway networks in Singapore and Portugal, using a dynamic traffic assignment model and a microscopic traffic simulator, respectively. Keywords: calibration algorithms; dynamic traffic assignment; microscopic traffic simulation; large–scale applications; optimisation; heuristic

Automatic Vehicle Trajectory Extraction by Aerial Remote Sensing

Research in road users’ behaviour typically depends on detailed observational data availability, particularly if the interest is in driving behaviour modelling. Among this type of data, vehicle trajectories are an important source of information for traffic flow theory, driving behaviour modelling, innovation in traffic management and safety and environmental studies. Recent developments in sensing technologies and image processing algorithms reduced the resources (time and costs) required for detailed traffic data collection, promoting the feasibility of site-based and vehicle-based naturalistic driving observation. For testing the core models of a traffic microsimulation application for safety assessment, vehicle trajectories were collected by remote sensing on a typical Portuguese suburban motorway. Multiple short flights over a stretch of an urban motorway allowed for the collection of several partial vehicle trajectories. In this paper the technical details of each step of the methodology used is presented: image collection, image processing, vehicle identification and vehicle tracking. To collect the images, a high-resolution camera was mounted on an aircraft's gyroscopic platform. The camera was connected to a DGPS for extraction of the camera position and allowed the collection of high resolution images at a low frame rate of 2s. After generic image orthorrectification using the flight details and the terrain model, computer vision techniques were used for fine rectification: the scale-invariant feature transform algorithm was used for detection and description of image features, and the random sample consensus algorithm for feature matching. Vehicle detection was carried out by median-based background subtraction. After the computation of the detected foreground and the shadow detection using a spectral ratio technique, region segmentation was used to identify candidates for vehicle positions. Finally, vehicles were tracked using a k- shortest disjoints paths algorithm. This approach allows for the optimization of an entire set of trajectories against all possible position candidates using motion-based optimization. Besides the importance of a new trajectory dataset that allows the development of new behavioural models and the validation of existing ones, this paper also describes the application of state-of-the-art algorithms and methods that significantly minimize the resources needed for such data collection. Keywords: Vehicle trajectories extraction, Driver behaviour, Remote sensin